Technical Field
The present invention relates to a control system to optimize imaging parameters of a digital video camera.
Related Background Art
High-speed digital cameras have become commonly used in toll collection and traffic law enforcement applications. These cameras must acquire images of fast moving objects with an image quality sufficient to identify the object(s) of interest. In the case of vehicles on a motorway, the objective is often to be able to identify the vehicle by clearly imaging the vehicle's license plate. Typically both the plate's alpha-numerics and the state name must be legible in the image. The larger alpha-numerics are often, but not always, in block print in a uniform color on a uniform or nearly uniform background. The state names however are often much smaller font sizes and printed in different colors and in various scripts. Generally modern license plates are designed more for aesthetics rather than to maximize legibility especially on those plates sporting multi-colored and/or scenic backgrounds. In order to recognize characters, a camera system must provide sufficient contrast between the lettering and the background. White license plates illuminated by direct sunlight (or strong reflections of nighttime lighting) generally yield sufficient video image contrast to easily recognize characters, however the illumination onto and therefore the radiance from such plates may easily be so high as to saturate the camera's sensor. In these cases the gain of the camera needs to be low enough to prevent image saturation. However portions of license plates that are in shadows (or during low nighttime illumination conditions) often lack enough video image contrast to support character recognition. In these cases the gain of the camera needs to be increased to create the required contrast for plate legibility. Imaging license plates therefore requires a means to allocate the dynamic range of the imaging sensor to simultaneously address both low light levels requiring increased gain and high light levels requiring decreased gain. To capture and make the video image of license plate characters and State name information legible requires high spatial resolution, a field of view (or multiple cameras) to observe at least one lane width, sufficient grey-scale/color contrast resolution to separate characters and State names from their background colors and sufficient dynamic range to prevent saturating the image whether the plate is in direct sunlight, in shadow, or under artificial illumination.
Recent advances in video camera technology have resulted in high spatial resolution sensors where pixels do not suffer from blooming or smearing problems (both of which can reduce or eliminate alpha-numeric contrast), have low noise so that small differences in alpha-numeric to local background contrasts can be resolved, and have gains that can vary across the array depending on the amount of light received (this feature supports imaging wide dynamic ranges without saturating). In order to provide sufficient contrast between characters and background of a license plate, a control system must set gain/exposure settings to make full use of the range of capabilities provided by these new sensors. The requirements to produce legible images are different in a high light situation versus a low light setting. For example, if a portion of a license plate is in shadow a high gain is needed to ensure good alpha-numeric to background contrast, whereas for a portion of a license plate in direct sunlight there is a naturally high contrast so that the gain can be lowered to prevent saturation of the image sensor and analog to digital electronics. A control system must be employed to make these tradeoffs.
The conditions under which the picture must be taken are also constantly changing. On short time scales the lighting during the daylight hours can change due to cloud cover and shadowing caused by other vehicles or even the vehicle that is being imaged. For typical traffic imaging applications the cameras are typically operational night and day and through all seasons of the year. The lighting conditions change as the angle of the sun changes relative to the plate surface during the course of a day and more slowly over the course of seasons. At night there is often artificial illumination provided to image the vehicle and its plate. Fixed exposure settings will not provide the image quality and brightness and contrast required to read the plate information.
For vehicles traveling at freeway speeds, the camera has only a fraction of a second to capture an image. License plates on vehicles may pass through the camera's field of view in ¼ second so there is little time for making camera gain controls during the time the vehicle's license plate is in the camera's field of view. In addition the first portion of the vehicle to appear in the camera's field of view may not be indicative of the irradiance levels seen at the end of the vehicle so first adjustments may not be best. Ideally the camera should be adjusted to take a good image of the vehicle and its license plate prior to both of them appearing in the camera's field of view.
Prior art gain control systems have relied upon using an external sensor to measure plate lighting conditions and define exposure settings. Other systems monitor the pixel values inside a portion of the camera's field of view to determine how to control camera gain. Both of these systems have drawbacks. External sensors add cost and complexity to the system and are often difficult to install in the optimum location to perform their light measurement function. Systems that use pixel value measurements for camera control suffer from time lags between when the light measurements are made versus when the camera gain control changes take affect and the relatively slow sampling rates of the pixel data relative to rapidly moving vehicles. In addition, the uncertainty of which pixel values correspond to actual license plate light levels (rather than the myriad of other radiance levels that can appear in the scene) means that the control input has a high uncertainty. Because of these issues, pixel value based control systems typically take several images in rapid succession at various gain settings to try to ensure that at least one results in a good image of the license plate. This may be difficult or impossible to achieve if the vehicle is moving at high speed and adds to the storage requirements of the system since multiple images must be saved until a determination as to which image is best for plate legibility.
There is a need for a control system for digital cameras that does not require light sensors (either internally using camera pixel values or externally using an auxiliary sensor) and yet the camera exposure controls are continuously adapted to ensure that a recognizable image of a vehicle and its license plate can be captured at the precise moment that the top and bottom of the vehicle and its license plate appear well framed within the camera's field of view. There is a need for a control system that does not suffer from any appreciable delays in setting proper camera controls or result in any significant delays in initiating a full frame image capture relative to the fastest expected rate of license plate movement through the camera's field of view. There is a need for a control system that ensures readability of both the numbers on the plates and the lettering that identifies the state of registration of the plate. There is a need for a system that can account for changing lighting conditions both on a short time scale (such as sudden cloud coverage of direct sunlight) and on long, even seasonal, time scales. There is a need for a system that can optimally take advantage of the full response range of the sensor. There is a need for a system that will provide sufficient contrast to read numbers, letters and otherwise identify an object in all conditions from low light levels where the object is shadowed to high light levels.